Data Mining Overview

Principles

Definition

Data mining is a multidisciplinary research field and methodological process concerned with the discovery of hidden patterns, insights, and knowledge from large datasets. It employs computational techniques, statistical models, and machine learning algorithms to reveal non-trivial structures and relationships, enabling informed decision-making and prediction across diverse domains.

Ontological type

Core Methods

Applications

Key Tasks

Key developments

Key Figures

Interpretable Model Foundations era

Fred Glover [1] of University of Colorado Boulder [3] and University of Colorado System [4] stands as a central figure in the Interpretable Model Foundations era. His Tabu Search—Part II [7] introduced a memory-based local search framework that improved solution quality and offered a transparent, rule-based path toward optimization central to this era. James H. Stock [2] affiliated with Harvard University [5] and Stanford University [6] contributed foundational ideas in interpretable modeling for these years. Testing for Common Trends [8] offered formal statistical tests for shared dynamics across time series, anchoring interpretable, model-based inference and enabling more reliable deployment of data-mining techniques in this era.

Scalable Pattern Mining era

Rakesh Agrawal[1] is a central figure in the Scalable Pattern Mining era, with affiliations at Cornell University[3] and ETH Zurich[4] during this period. His key contributions in this era include Mining association rules between sets of items in large databases[7], Fast algorithms for mining association rules[8], and Fast Algorithms for Mining Association Rules in Large Databases[9], which collectively advanced scalable rule discovery by drastically reducing candidate generation. Jiawei Han[2] is another leading figure linked to the University of Hong Kong[5] and the University of Illinois Urbana-Champaign[6] in this era. Han's pivotal works—Data mining: concepts and techniques[10], Mining frequent patterns without candidate generation[11], and Exploratory mining and pruning optimizations of constrained associations rules[12]—shaped foundational methods and guided the development of constraint-guided and scalable mining in large datasets.

Embedding-Centric Mining era

Ali A. Ghorbani [1] is associated with Nanyang Technological University [3] and the University of Hong Kong [4] in this era. His key contributions in this era, based on Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization [7], involve generating a new intrusion detection dataset and characterizing intrusion traffic, addressing data scarcity and benchmarking needs for security analytics in embedding-centric pipelines. Zhu Han [2] is associated with Shanghai Medical College of Fudan University [5] and the University of California, San Diego [6] in this era. Contributions in this era, based on Deep Interest Network for Click-Through Rate Prediction [8], involve developing Deep Interest Network models for CTR prediction, enabling more accurate and personalized recommendations in large-scale embedding-centric systems.